A simultaneous perturbation weak derivative estimator for stochastic neural networks
نویسندگان
چکیده
منابع مشابه
Simultaneous perturbation stochastic approximation of nonsmooth functions
A simultaneous perturbation stochastic approximation (SPSA) method has been developed in this paper, using the operators of perturbation with the Lipschitz density function. This model enables us to use the approximation of the objective function by twice differentiable functions and to present their gradients by volume integrals. The calculus of the stochastic gradient by means of this present...
متن کاملMixed two-stage derivative estimator for sensitivity analysis
In mathematical modeling, determining most influential parameters on outputs is of major importance. Thus, sensitivity analysis of parameters plays an important role in model validation. We give detailed procedure of constructing a new derivative estimator for general performance measure in Gaussian systems. We will take advantage of using score function and measure-value derivative estimators ...
متن کاملTrace formulas for stochastic evolution operators: Weak noise perturbation theory
Periodic orbit theory is an effective tool for the analysis of classical and quantum chaotic systems. In this paper we extend this approach to stochastic systems, in particular to mappings with additive noise. The theory is cast in the standard field theoretic formalism, and weak noise perturbation theory written in terms of Feynman diagrams. The result is a stochastic analog of the next-to-lea...
متن کاملSimultaneous perturbation stochastic approximation for automatic speech recognition
While both the acoustic model and the language model in automatic speech recognition are typically well-trained on the target domain, the free parameters of the decoder itself are often set manually. In this paper, we investigate in how far a stochastic approximation algorithm can be employed to automatically determine the best parameters, especially if additional timeconstraints are given on u...
متن کاملAdaptive initial step size selection for Simultaneous Perturbation Stochastic Approximation.
A difficulty in using Simultaneous Perturbation Stochastics Approximation (SPSA) is its performance sensitivity to the step sizes chosen at the initial stage of the iteration. If the step size is too large, the solution estimate may fail to converge. The proposed adaptive stepping method automatically reduces the initial step size of the SPSA so that reduction of the objective function value oc...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Computational Management Science
سال: 2019
ISSN: 1619-697X,1619-6988
DOI: 10.1007/s10287-019-00357-1